Multi-kernel Passive Stochastic Gradient Algorithms and Transfer Learning

This paper develops a novel passive stochastic gradient algorithm. In approximation, the algorithm does not have control over location where noisy gradients of cost function are evaluated. Classical algorithms use kernel that approximates Dirac delta to weigh based on how far they evaluated from desired point. this we construct multi-kernel The performs substantially better in high dimensional problems and incorporates variance reduction. We analyze weak convergence its rate convergence. numerical examples, study version least mean squares (LMS) for transfer learning compare performance with classical version.